{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Initializing the NUTS-HMC sampler\n", "\n", "In this example, we show how to use parameter estimates returned by any of Stan's inference algorithms as initial values for Stan's NUTS-HMC sampler. These include:\n", "\n", "* [Pathfinder ](https://mc-stan.org/docs/cmdstan-guide/pathfinder-config.html) \n", "* [ADVI ](https://mc-stan.org/docs/cmdstan-guide/variational_config.html) \n", "* [Laplace](https://mc-stan.org/docs/cmdstan-guide/laplace_sample_config.html)\n", "* [Optimization](https://mc-stan.org/docs/cmdstan-guide/optimize_config.html)\n", "* [NUTS-HMC MCMC](https://mc-stan.org/docs/cmdstan-guide/mcmc_config.html)\n", "\n", "By default, the NUTS-HMC sampler randomly initializes all (unconstrained) model parameters uniformly in the interval (-2, 2). If this interval is far from the typical set of the posterior, initializing sampling from these approximation algorithms can speed up and improve adaptation.\n", "\n", "### Model and data\n", "\n", "The Stan model and data are taken from the [posteriordb package](https://github.com/stan-dev/posteriordb).\n", "\n", "We use the [blr model](https://github.com/stan-dev/posteriordb/blob/master/posterior_database/models/stan/blr.stan),\n", "a Bayesian standard linear regression model with noninformative priors,\n", "and its corresponding simulated dataset [sblri.json](https://github.com/stan-dev/posteriordb/blob/master/posterior_database/data/data/sblri.json.zip),\n", "which was simulated via script [sblr.R](https://github.com/stan-dev/posteriordb/blob/master/posterior_database/data/data-raw/sblr/sblr.R).\n", "For convenience, this example assumes the posteriordb model and data are local, in files `blr.stan` and `sblri.json`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "from cmdstanpy import CmdStanModel\n", "\n", "stan_file = 'blr.stan' # basic linear regression\n", "data_file = 'sblri.json' # simulated data\n", "\n", "model = CmdStanModel(stan_file=stan_file)\n", "\n", "print(model.code())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Demonstration with Stan's `pathfinder` method\n", "\n", "Initializing the sampler with estimates from any previous inference algorithm follows the same general usage pattern. First, we call the \n", "corresponding method on the `CmdStanModel` object. From the resulting fit, we call the `.create_inits()` \n", "method to construct a set of per-chain initializations for the model parameters. To make it explicit, \n", "we will walk through the process using the `pathfinder` method (which wraps the \n", "CmdStan [pathfinder ](https://mc-stan.org/docs/cmdstan-guide/pathfinder-config.html) method).\n", "\n", "Pathfinder locates normal approximations to the target\n", "density along a quasi-Newton optimization path, with local covariance\n", "estimated using the negative inverse Hessian estimates produced by the\n", "LBFGS optimizer. Pathfinder returns draws from the Gaussian approximation\n", "with the lowest estimated Kullback-Leibler (KL) divergence to the true\n", "posterior.\n", "By default, CmdStanPy runs multi-path Pathfinder which returns an importance-resampled \n", "set of draws over the outputs of 4 independent single-path Pathfinders.\n", "This better matches non-normal target densities and also mitigates\n", "the problem of L-BFGS getting stuck at local optima or in saddle points on plateaus.\n", "\n", "We obtain Pathfinder estimates by calling the `.pathfinder()` method which returns a `CmdStanPathfinder` object:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pathfinder_fit = model.pathfinder(data=data_file, seed=123)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Posteriordb provides reference posteriors for all models. For the blr model, conditioned on the dataset `sblri.json`, the reference posteriors can be found in the [sblri-blr.json](https://github.com/stan-dev/posteriordb/blob/master/posterior_database/reference_posteriors/summary_statistics/mean/mean/sblri-blr.json) file.\n", "\n", "The reference posteriors for all elements of `beta` and `sigma` are all very close to 1.0.\n", "\n", "The experiments reported in Figure 3 of the paper [Pathfinder: Parallel quasi-Newton variational inference](https://arxiv.org/abs/2108.03782) by Zhang et al. show that Pathfinder provides a better estimate of the posterior, as measured by the 1-Wasserstein distance to the reference posterior, than 75 iterations of the warmup Phase I algorithm used by the NUTS-HMC sampler.\n", "Furthermore, Pathfinder is more computationally efficient, requiring fewer evaluations of the log density and gradient functions. Therefore, using the Pathfinder estimates to initialize the parameter values for the NUTS-HMC sampler can allow the sampler to do a better job of adapting the stepsize and metric during warmup, resulting in better performance and estimation.\n", "\n", "We construct the parameter inits for full MCMC sampling below. The `.create_inits()` default behavior is to create inits for four chains to correspond with the sampling defaults. You can requests more or less by modifying the `chains` keyword argument." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pathfinder_inits = pathfinder_fit.create_inits()\n", "for chain_init in pathfinder_inits:\n", " print(chain_init)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that the Pathfinder inits are close the reference posteriors for the parameters. To use these inits, we pass the `pathfinder_inits` object to the `inits` kwarg:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "mcmc_pathfinder_inits_fit = model.sample(\n", " data=data_file, inits=pathfinder_inits, iter_warmup=75, seed=12345\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(mcmc_pathfinder_inits_fit.diagnose())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Despite only running 75 warmup iterations, all posterior diagnostics from the sampler look good." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "mcmc_pathfinder_inits_fit.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we were to instead use the default random parameter initializations, we would need to run more warmup iterations to produce useful samples. For example, if we only run the same 75 warmup iterations with random inits, the result fails to estimate `sigma` correctly: " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "mcmc_random_inits_fit = model.sample(data=data_file, iter_warmup=75, seed=12345)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "mcmc_random_inits_fit.summary()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(mcmc_random_inits_fit.diagnose())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The diagnostics clearly indicate problems with estimating `sigma`. In this case, it is necessary to run the model with at least 150 warmup iterations to produce a good set of estimates when starting from the default initialization." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Other inference algorithms" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can follow the same pattern with Stan's ADVI algorithm by first using the `CmdStanModel.variationl` method. Because this algorithm is unstable and may fail to converge, we run it with argument `require_converged` set to `False`. We also specify a seed, to avoid instabilities as well as for reproducibility." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "vb_fit = model.variational(data=data_file, require_converged=False, seed=123)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The ADVI algorithm provides estimates of all model parameters.\n", "\n", "The `variational` method returns a `CmdStanVB` object, which similarly can construct a set of inits with the `.create_inits()` method:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "vb_inits = vb_fit.create_inits()\n", "for chain_init in vb_inits:\n", " print(chain_init)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Which can be passed to the `inits` keyword argument of the sample method." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "mcmc_vb_inits_fit = model.sample(\n", " data=data_file, inits=vb_inits, iter_warmup=75, seed=12345\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(mcmc_vb_inits_fit.diagnose())" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "mcmc_vb_inits_fit.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The sampler estimates match the reference posterior with no diagnostic issues." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Inits can also be constructed from the `laplace` method:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "laplace_inits = model.laplace_sample(data=data_file, seed=123).create_inits()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "mcmc_laplace_inits_fit = model.sample(\n", " data=data_file, inits=laplace_inits, iter_warmup=75, seed=12345\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(mcmc_laplace_inits_fit.diagnose())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And the `optimize` method. Since optimizations attempts to return a posterior mode, this will initialize all chains at the same point, which is not typically ideal." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "optimized_inits = model.optimize(data=data_file, seed=123).create_inits()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "mcmc_optimize_inits_fit = model.sample(\n", " data=data_file, inits=optimized_inits, iter_warmup=75, seed=12345\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(mcmc_optimize_inits_fit.diagnose())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is also possible to use the output of the `sample()` method itself to construct inits to be fed into a future sampling run:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "first_mcmc_inits = model.sample(\n", " data=data_file, iter_warmup=75, seed=12345\n", ").create_inits()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "second_mcmc_fit = model.sample(data=data_file, inits=first_mcmc_inits, iter_warmup=75, seed=12345)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(second_mcmc_fit.diagnose())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that despite the initial sampling issues in the first MCMC run, the inits sourced from that run result in reasonable sampling in the second run." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.12" } }, "nbformat": 4, "nbformat_minor": 4 }